Integrating Reinforcement Learning with Soar
نویسندگان
چکیده
In this paper, we describe an architectural modification to Soar that gives a Soar agent the opportunity to learn statistical information about the past success of its actions and utilize this information when selecting an operator. This mechanism serves the same purpose as production utilities in ACT-R, but the implementation is more directly tied to the standard definition of the reinforcement learning (RL) problem. The paper explains our implementation, gives a rationale for adding an RL capability to Soar, and shows results for SoarRL agents’ performance on two tasks. Long-term Procedural Memory Production Rules Short-term Declarative Memory Decision Procedure
منابع مشابه
Soar-RL: integrating reinforcement learning with Soar
In this paper, we describe an architectural modification to Soar that gives a Soar agent the opportunity to learn statistical information about the past success of its actions and utilize this information when selecting an operator. This mechanism serves the same purpose as production utilities in ACT-R, but the implementation is more directly tied to the standard definition of the reinforcemen...
متن کاملA Case Study in Integrating Probabilistic Decision Making and Learning in a Symbolic Cognitive Architecture: Soar Plays Dice
One challenge for cognitive architectures is to effectively use different forms of knowledge and learning. We present a case study of Soar agents that play a multiplayer dice game, in which probabilistic reasoning and heuristic symbolic knowledge appear to play a central role. We develop and evaluate a collection of agents that use different combinations of probabilistic decision making, heuris...
متن کاملInstance-Based Online Learning of Deterministic Relational Action Models
We present an instance-based, online method for learning action models in unanticipated, relational domains. Our algorithm memorizes preand post-states of transitions an agent encounters while experiencing the environment, and makes predictions by using analogy to map the recorded transitions to novel situations. Our algorithm is implemented in the Soar cognitive architecture, integrating its t...
متن کاملInvestigating the Soar-RL Implementation of the MAXQ Method for Hierarchical Reinforcement Learning
Discussed in greater detail below, Soar-RL is the integration of the reinforcement learning method of machine learning into Soar, a generalized architecture. The MAXQ method for hierarchical reinforcement learning [1] greatly influenced the design for the hierarchical reinforcement learning components of Soar-RL [2]. In its pre-release form, it is prudent to question the merits of this union: w...
متن کاملEmotion-Driven Reinforcement Learning
Existing computational models of emotion are primarily concerned with creating more realistic agents, with recent efforts looking into matching human data, including qualitative emotional responses and dynamics. In this paper, our work focuses on the functional benefits of emotion in a cognitive system where emotional feedback helps drive reinforcement learning. Our system is an integration of ...
متن کامل